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August  7,  2001 

The  Honorable  Gordon  R.  England 
The  Secretary  of  the  Navy 

Dear  Mr.  Secretary: 

During  our  review  of  the  Navy’s  Space  and  Naval  Warfare  (SPAWAR) 
Systems  Command’s  fiscal  year  2001  budget  request,  we  found  that  many 
information  technology  systems  were  being  procured  and  fielded  in 
relatively  large  quantities — sometimes  exceeding  50  percent  of  the  total — 
during  low-rate  initial  production  and  before  completing  operational 
testing.  The  primary  purpose  of  low-rate  initial  production  is  to  produce 
enough  units  for  operational  testing  and  evaluation  and  to  establish 
production  capabilities  to  prepare  for  full-rate  production.  limiting  initial 
production  affords  the  opportunity  to  confirm  the  stability  and  soundness 
of  a  new  system  before  producing  larger  quantities.  Commercial  and 
Department  of  Defense  (DOD)  best  practices  have  shown  that  completing 
a  system’s  testing  prior  to  producing  significant  quantities  will 
substantially  lower  the  risk  of  costly  fixes  and  retrofits.  For  major 
weapons  systems,  there  are  statutory’  provisions  that  limit  the  quantities 
of  systems  produced  dining  low-rate  initial  production  to  the  minimum 
quantity  necessary  and  require  justification  for  quantities  exceeding 
10  percent  of  total  production.  Although  these  provisions  do  not  apply  to 
non-major  systems,  DOD  and  Navy  acquisition  regulations  encourage 
these  programs  to  make  use  of  the  low-rate  initial  production  concept 
where  appropriate. 

Given  the  risks  associated  with  acquiring  high  percentages  of  production 
quantities  before  completing  operational  testing,  we  reviewed  the 
acquisitions  of  information  systems  at  SPAWAR  to  determine  (1)  why 
information  systems  were  being  procured  and  fielded  in  large  numbers 
before  operational  testing,  (2)  what  effects  this  practice  was  having  on 
SPAWAR  and  the  fleet,  and  (3)  what  the  Navy  is  doing  to  mitigate  the  risks 
associated  with  this  practice. 
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Results  in  Brief 


According  to  SPAWAR  officials,  the  main  reason  for  the  high  percentage 
of  low-rate  initial  production  quantities  is  to  more  quickly  respond  to  fleet 
demands  for  information  systems  improvements.  The  SPAWAR 
commander  noted  that  the  high  percentage  reflects  the  desire  to  quickly 
field  information  systems  to  keep  up  with  rapid  advances  in  technology. 

He  further  noted  that  the  high  percentage  of  low-rate  initial  production 
buys  is  generally  considered  low  risk  when  they  involve  proven 
commercial  technology  and  are  low-cost  items — compared  to  ships  and 
aircraft.  Officials  in  the  Office  of  the  Chief  of  Naval  Operations  said  the 
fleet  would  rather  have  an  information  system  with  75  to  80  percent 
capability  than  to  wait  until  SPAWAR  completes  all  necessary  testing  to 
ensure  a  system  will  meet  all  of  its  capability. 

Many  information  technology  systems  purchased  and  fielded  during  low- 
rate  initial  production  and  prior  to  completing  operational  testing 
experienced  problems  that  negatively  impacted  fleet  operations  and 
capabilities.  We  analyzed  8  of  the  21  fielded  SPAWAR  programs  with  a 
high  percentage  of  low-rate  initial  production  quantities  and  found  that  all 
had  performance  problems,  all  had  interoperability  problems,  and  6  had 
suitabiUty  problems.  A  limited  analysis  of  the  remaining  13  systems  found 
that  several  also  experienced  operational  problems.  The  SPAWAR 
commander  said  most  of  the  problems  we  found  were,  in  part,  the  result  of 
an  acquisition  process  that  needed  more  discipline  and  has  recently 
improved.  He  further  noted  that  some  operational  problems  with  new 
systems  are  inevitable  and  are  part  of  the  cost  of  doing  business  and  are 
worth  the  risk  in  order  to  provide  systems  to  the  fleet  quickly. 
Nevertheless,  the  DOD  Director  of  Operational  Test  and  Evaluation  said 
that  low-rate  initial  items  can  be  used  in  an  operational  environment  to 
learn  about  problems  and  fix  them,  but  procuring  high  percentages  of  a 
system  before  completing  operational  test  and  evaluation  is  a  risky 
strategy. 

SPAWAR  has  taken  a  number  of  steps  to  mitigate  the  risks  of  high 
percentage  low-rate  initial  production  procurements.  To  add  more 
discipline  and  rigor  to  the  low-rate  initial  production  decision  process,  the 
Command  now  requires  program  managers  to  use  a  standardized  checklist 
and  report  template  as  part  of  reviewing  and  approving  low-rate  initial 
production  purchase  requests.  SPAWAR  has  also  established  an 
Acquisition  Reform  Office  to  serve  as  a  focal  point  and  command-wide 
disseminator  of  lessons  learned  and  process  improvements.  The  reform 
office  is  currently  developing  a  “Rules  of  the  Road”  Acquisition  Guidebook 
for  SPAWAR  program  managers.  Further,  in  discussions  of  our  findings 
and  observations  during  this  review,  the  Navy  and  SPAWAR  agreed  to 
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Background 


provide  better  risk  management  guidance  for  information  systems.  They 
also  agreed  to  our  suggested  improvements  in  documenting,  justifying,  and 
reporting  low-rate  initial  production  decisions. 

In  commenting  on  our  draft  report,  the  Navy  agreed  and  stated  that  the 
actions  taken  and  planned  by  it  and  the  SPAWAR  Command  are  expected 
to  improve  the  Navy’s  low-rate  initial  production  decision  process.  The 
Navy’s  comments  appear  in  appendix  I. 


SPAWAR  is  one  of  the  Navy’s  three  major  acquisition  commands. 

SPAWAR  provides  information  technology  systems  to  naval  forces  on 
land,  at  sea,  and  in  space  and  integrates  all  information  products, 
including  those  developed  by  other  systems  commands  and  agencies 
outside  the  Navy.  The  SPAWAR  workforce  is  over  7,300  personnel,  and  the 
fiscal  year  2000  budget  was  $3.7  billion — about  $2.7  billion  to  develop  and 
procure  systems  and  about  $1.0  billion  to  operate  and  maintain  them 
Specifically,  SPAWAR  develops,  acquires,  and  manages 

•  battle  management  systems  (for  example,  software  applications  and 
computers); 

•  undersea,  terrestrial,  and  space  sensors  (for  example,  underwater 
sensors,  navigation  and  weather  systems,  and  satellites); 

•  information  transfer  systems  (for  example,  communications  systems, 
radios,  antennas,  and  switches);  and 

•  information  management  systems  (for  example,  local  area  networks 
and  routers). 

As  of  October  2000,  SPAWAR  was  managing  21  programs  that  involved 
low-rate  initial  production.  These  programs  had  cumulative  low-rate  initial 
procurements  ranging  fi’om  5  to  100  percent  of  the  total  inventory 
objective — 7  systems  were  above  50  percent.  'The  estimated  total 
production  costs  for  the  eight  systems  we  analyzed  in  detail  ranged  from 
$31  million  to  $525  million. 

As  weapon  system  programs  move  through  the  phases  of  the  acquisition 
process,  they  are  subject  to  review  at  major  decision  points  called 
milestones.  M^or  defense  acquisition  programs,  known  as  acquisition 
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category  I,  and  major  systems  programs,  known  as  acquisition  category 
as  well  as  non-major  systems  programs,  known  as  acquisition  category  III 
and  IV,®  follow  the  same  general  process.  DOD  and  Navy  acquisition 
policies  state  that  program  risks  shall  be  assessed  at  each  milestone 
decision  point  before  approval  is  granted  for  the  next  phase.  The  policies 
add  that  test  and  evaluation  shall  be  used  to  determine  system  maturity 
and  identify  areas  of  technical  risk. 

Major  milestones  in  DOD’s  systems  acquisition  process  include 

•  Milestone  0,  when  the  determination  is  made  about  whether  an 
identified  mission  need  warrants  a  study  of  alternative  concepts  to 
satisfy  the  need.  If  warranted,  the  program  is  approved  to  begin  the 
concept  exploration  and  definition  phase. 

•  Milestone  I,  when  the  determination  is  made  about  whether  a  new 
acquisition  program  is  warranted.  If  so,  initial  cost,  schedule,  and 
performance  goals  are  established  for  the  program,  and  authorization  is 
given  to  start  the  demonstration  and  validation  phase. 

•  Milestone  H,  when  the  determination  is  made  about  whether 
continuation  of  development,  testing,  and  preparation  for  production  is 
warranted.  If  so,  authorization  is  given  to  start  the  engineering  and 
manufacturing  development  phase.  Approval  of  this  phase  will  often 
involve  a  commitment  to  low-rate  initial  production,  which  is  defined 
as  the  minimum  quantify  needed  to  (1)  provide  production- 
representative  articles  for  operational  testing  and  evaluation, 

(2)  establish  an  initial  production  base,  and  (3)  permit  orderly  ramp-up 
to  full-rate  production  upon  completion  of  operational  testing  and 
evaluation.  Operational  test  and  evaluation  is  a  key  internal  control  to 
ensure  that  decisionmakers  have  objective  information  available  on  a 


®  The  Navy  defines  acquisition  category  I  as  a  mqjor  defense  acquisition  program  that  is 
estimated  to  cost  more  than  $355  million  (fiscal  year  1996  dollars)  for  research, 
development,  testing,  and  evaluation  or  more  than  $2. 1  billion  (fiscal  year  1996  dollars)  for 
procurement.  The  Navy  defines  acquisition  category  n  as  a  mqjor  system  program  that  is 
estimated  to  cost  more  than  $140  miUion  (fiscal  year  1996  dollars)  for  research, 
development,  testing,  and  evaluation  or  more  than  $645  million  (fiscal  year  1996  dollars) 
for  procurement. 

®  The  Navy  defines  acquisition  category  III  as  a  weapon  system  program  not  designated 
acquisition  category  I  or  H  and  which  affects  the  military  characteristics  of  ships  or  aircraft 
or  involves  combat  capability.  The  Navy  defines  acquisition  category  IV  as  those  not 
designated  as  acquisition  category  I,  n,  or  IE. 
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weapon  system’s  performance  to  minimize  risks  of  procuring  costly 
and  ineffective  systems.  Operational  testing  and  evaluation  uses  field 
tests  under  realistic  conditions  to  determine  the  operational 
effectiveness^  and  suitability®  of  a  system  for  use  in  combat.  DOD 
acquisition  regulations  generally  provide  that  programs  successfully 
complete  these  tests  before  starting  full-rate  production. 

•  Milestone  HI,  when  operational  test  and  evaluation  has  been 

completed,  a  determination  is  made  about  whether  to  proceed  to  fuU- 
rate  production  and  field  the  system. 

Over  the  years,  we  have  foimd  instances  in  which  DOD  used  the  low-rate 
initial  production  decision  phase  to  purchase  significant  numbers  of  major 
and  non-mqjor  systems  without  successfully  completing  operational 
testing  and  evaluation.  Often,  these  systems  later  experienced  significant 
effectiveness  and/or  suitability  problems.  In  1994,  we  reported  that  DOD 
had  made  large  buys  of  weapon  systems  during  the  low-rate  initial 
production  phase  and  prior  to  completion  of  operational  test  and 
evaluation,  which  resulted  in  operational  problems.®  For  example,  we 
reported  that  the  Navy  procured  100  percent  of  a  system’s  inventory 
objective  during  low-rate  initial  production  and  later  found  that  the  system 
lacked  critical  hardware  and  software  capabilities.  In  another  case,  the 
Navy  procured  100  percent  of  a  system’s  inventory  objective  during  low- 
rate  initial  production  and  later  terminated  the  program  when  it  failed 
operational  testing  and  evaluation.  A  recent  Defense  Science  Board  report 
fovmd  that  weapons  systems  are  stiU  being  fielded  without  adequate 
testing  to  assure  their  effectiveness  and  utility  to  operating  units.^  We 
conducted  this  review  because  buying  systems  before  completing 
operational  testing  has  inherent  risks,  and  SPAWAR’s  practice  of  buying 


^  DOD  defines  operational  effectiveness  as  a  system’s  overall  degree  of  mission 
accomplishment  when  representative  personnel  use  the  system  in  its  planned  or  expected 
operational  environment  considering  organization,  doctrine,  tactics,  survivability, 
vulnerability,  and  threat. 

®  DOD  defines  operational  suitability  as  the  degree  to  which  a  system  can  be  satisfactorily 
placed  in  field  use  considering  such  factors  as  availability,  compatibility,  transportability, 
interoperability,  reliability,  wartime  usage  rates,  maintainability,  safety,  and  supportability. 

®  Weapons  Acquisition:  Low-Rate  Initial  Production  Used  to  Buy  Weapon  Systems 
/^emarure{y(GAO/NSIAD-95-18,  Nov.  24, 1994). 

Report  of  the  Defense  Science  Board  Task  Force  on  Test  and  Evaluation  Capabilities, 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Dec.  2000. 
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high  percentages  of  a  system’s  total  inventory  objective  during  low-rate 
initial  production  raised  these  risk  concerns. 


Fleet  Demands  and 
Rapidly  Advancing 
Technology  Cited  for 
the  High  Percentage 
of  Low-Rate  Initial 
Production  Purchases 


SPAWAR  officials  cited  three  primary  reasons  for  high-percentage  buys 
during  low-rate  initial  production.  First,  to  meet  Navy  initiatives,  SPAWAR 
must  provide  the  fleet  with  large  quantities  of  information  technology 
systems  as  quickly  as  possible.  Second,  many  information  systems  consist 
of  evolving  technology  that  quickly  becomes  obsolete.  Third,  additional 
low-rate  initial  production  buys  are  approved  due  to  delays  in  conducting 
the  operational  test  and  evaluation. 

The  main  reason  that  SPAWAR  officials  cited  for  hi^-percentage  buys 
during  low-rate  initial  production  is  the  need  to  provide  as  many 
information  technology  systems  to  the  fleet  as  quickly  as  possible  to  meet 
several  Navy  initiatives.  The  Navy’s  current  vision  for  the  21st  century. 
Forward  From  The  Sea,  involves  innovations  in  technology  to  rapidly 
transform  the  Navy  into  a  21st  century  force.  SPAWAR  provides  or 
contributes  to  many  of  the  operational  capabilities  that  support  the  vision. 
Officials  in  the  Chief  of  Naval  Operations’  Fleet  and  AlUed  Requirements 
Division  stated  that  the  fleets  put  pressure  on  SPAWAR  to  provide 
information  systems  faster.  These  officials,  as  well  as  SPAWAR  officials, 
contend  that  if  SPAWAR  does  not  provide  systems  to  the  fleet  quickly, 
then  the  fleet  will  bypass  the  Chief  of  Naval  Operations  and  SPAWAR  and 
procure  some  systems  with  fleet  funding.  If  the  fleet  buys  the  systems, 
SPAWAR  cannot  control  the  configuration  of  these  systems,  which  can 
eventually  result  in  interoperability  problems  with  systems  that  SPAWAR 
procures.  An  official  in  the  Office  of  the  Deputy  Assistant  Secretary  of  the 
Navy  responsible  for  communications  and  space  systems  also  agreed  that 
there  is  pressure  on  SPAWAR  to  meet  fleet  demands.  He  said  that  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
stated  about  3  years  ago  that  the  pace  of  developing  systems  was  too  slow 
and  called  for  shortening  the  development  cycle  by  incorporating 
evolutionary  development  and  acquisition.  Through  evolutionary 
development  and  acquisition,  systems  are  continuously  improved,  as  new 
technology  becomes  available. 


According  to  the  SPAWAR  commander,  another  reason  for  quickly 
providing  systems  to  the  fleet  is  that  information  systems  consist  of 
rapidly  advancing  technology,  which  can  become  obsolete  within 
18  months.  He  said  that  procuring  and  fielding  a  large  percentage  of  a 
system’s  inventory  objective  while  still  in  low-rate  initial  production 
quickly  provide  the  fleet  with  better  information  systems  and  provide 
important  operational  data  so  that  any  system  performance  problems  can 
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be  quickly  fixed.  Officials  in  the  Navy  Fleet  and  Allied  Requirements 
Division  said  that  the  18-month  obsolescence  cycle  is  the  main  reason  that 
the  fleet  would  rather  have  a  system  now  with  75  to  80  percent  of  its  full 
capability  as  opposed  to  waiting  until  the  system  has  all  of  its  capability- 
However,  officials  in  the  Chief  of  Naval  Operations’  Office  of  Test, 
Evaluation,  and  Technology  Requirements  disputed  that  rapidly  advancing 
technology  is  a  legitimate  reason  for  making  hi^-percentage  buys  during 
low-rate  initial  production  and  before  completing  operational  testing. 
These  officials  concluded  that  making  high-percentage  buys  during  low- 
rate  initial  production  circumvents  the  operational  testing  and  evaluation 
process  and  increases  the  risk  that  systems  will  not  work  as  intended 
when  fielded. 

High-percentage  buys  during  low-rate  initial  production  also  were  the 
result  of  delays  in  conducting  the  operational  test  and  evaluation. 
According  to  the  SPAWAR  commander,  the  pass  or  fail  nature  of 
operational  testing  contributed  to  delays.  Rather  than  fail  a  test  and  risk 
program  reduction  or  termination,  operational  tests  were  delayed  until 
there  was  a  good  probability  that  the  system  would  pass  the  tests.  As  tests 
were  delayed,  additional  low-rate  initial  production  buys  were  approved. 
Six  of  the  eight  systems  we  analyzed  had  additions  to  the  original  low-rate 
initial  production  quantities. 

The  SPAWAR  commander  said  that  making  high-percentage  buys  of  a 
system  while  still  in  low-rate  initial  production  is  low  risk  when  proven 
commercial  technology  items  are  being  procured  and  they  are  relatively 
low-cost  items — when  compared  to  the  cost  of  ships  and  aircraft.  Further, 
if  problems  arise  after  the  low-rate  initial  production  systems  are  fielded, 
the  cost  to  fix  them  is  not  significant.  However,  SPAWAR  officials  agreed 
that  none  of  the  eight  systems  we  analyzed  are  entirely  commercial  and 
that  all  of  them  have  military  requirements  that  must  be  tested  in  a 
realistic  operational  environment. 

The  DOD  Director  of  Operational  Test  and  Evaluation  agreed  that  low-rate 
initial  production  items  can  be  used  in  an  operational  environment  to  learn 
about  problems  and  fix  them,  but  he  said  that  procuring  large  quantities  of 
a  system  before  operational  test  and  evaluation  is  a  risky  strategy. 
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Problems 

Experienced  When 
Procuring  High 
Percentages  of  Units 
In  Low-Rate  Initial 
Production 


We  analyzed  eight  SPAWAE  systems  and  fovind  seven  of  them  had  a 
combination  of  problems  that  adversely  affected  fleet  operations — all  had 
performance  problems,  all  had  interoperability  problems,  and  six  had 
suitability  problems.  A  performance  problem  is  the  inability  of  a  system  to 
effectively  and  efficiently  perform  its  assigned  mission.  An  interoperability 
problem  is  the  inability  of  systems  to  work  together  effectively  to  provide 
services  to  and  accept  services  from  other  systems.  A  suitability  problem 
involves  a  system  not  satisfactorily  meeting  one  or  more  requirements, 
including  reliability,  maintainability,  logistics  support,  or  training.  These 
problems  may  delay  progress  in  achieving  the  Navy’s  vision  for  using 
information  technology  to  attain  and  maintain  network-centric  warfighting 
knowledge  and  decision-making  superiority.  The  last  of  the  eight  systems 
had  not  been  installed  when  we  analyzed  the  eight  systems.  Table  1 
illustrates  the  types  of  problems  identified  for  seven  of  the  eight  systems. 


Table  1 :  Types  of  Problems  for  High-Percentage  Low-rate  Initial  Production  Systems 

System 

Current 

Percent _ 

Types  of  problems 

Name 

Description 

inventory 

objective 

1  ow-rate  1  nteroper- 

oroduction  Performance  ability 

Suitability 

Automated  Digital  Network 
System/  Sensitive 
Compartmented 

Information  Automated 
Dipital  Network  System 

Provides  communications 
network  and  selects  best 
path  for  transmissions 

513 

51 

X 

X 

X 

Combat  Direction  Finding 
System 

Detects,  identifies,  and 
provides  long-range 
hostile  target  data 

36 

22 

X 

X 

Command  and  Control 
Processor 

Provides  interface  for 
communications  and 
shipboard  processors 

105 

41 

X 

X 

X 

Commercial  Wideband 
Satellite  Program 
(Terminals) 

Provides  super-high 
frequency  ship  satellite 
communications 

37 

78 

X 

X 

X 

Digital  Wideband 
Transmission  System 

Provides  ultra-high 
frequency  voice,  video, 
and  data  communications 

45 

100 

X 

X 

X 

Miniaturized  Demand 
Assigned  Multiple  Access 

Provides  capability  to 
share  multiple  signals  on 
a  satellite  broadcast 

142 

72 

X 

X 

X 

Navy  Modular  Automated 
Communications  System  II 

Provides  automatic 
processing  and  storage  of 
messages 

305 

21 

X 

X 

X 

Submarine  High  Data  Rate 

Provides  high  capacity 
communications  In 
extremely  and  super  high 
frequencies 

60 

37 

a 

a 

“  The  first  low-rate  initial  production  unit  had  not  been  installed  as  of  September  2000;  therefore,  no 
information  existed  about  its  operational  effectiveness  and  suitability  in  the  fleet. 
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Two  of  the  systems — ^the  Digital  Wideband  Transmission  System  and  the 
Command  and  Control  Processor — illustrate  how  interoperability, 
performance,  and  suitability  problems  impact  fleet  operations.  The  Digital 
Wideband  Transmission  System  is  a  radio  transmission  system  supporting 
voice,  video,  and  data  communications.  It  is  required  on  all  aircraft 
carriers  and  amphibious  ships  and  at  training  facilities.  SPAWAE  approved 
100  percent  of  the  total  inventory  objective  to  be  bought  under  low-rate 
initial  production;  however,  the  system  does  not  work  due  to  a  number  of 
problems  with  the  antenna,  power  amplifier,  and  radio  fi"equency  control. 
The  system  also  created,  and  was  affected  by,  electromagnetic 
interference,  which  caused  severe  interoperability  problems.  For  example, 
the  interference  caused  a  complete  loss  of  Global  Positioning  System 
navigation  capability.  Furthermore,  in  order  for  the  digital  system  to  work, 
an  air-search  radar  had  to  be  shut  down.  Consequently,  the  Navy  turned 
off  the  system  the  first  time  it  was  used  during  an  amphibious  ready  group 
deployment  in  the  Pacific,  replaced  it  with  a  legacy  system,  and  placed  the 
digital  system  in  an  inoperative  status  for  9  months.  By  November  2000, 
SPA  WAR  had  installed  78  percent  of  the  systems  in  the  fleet,  but  it  will 
cost  $4.3  million  to  fix  the  problems  for  this  $40-million  program — 

$1.2  million  for  engineering  work  and  $3.1  million  for  retrofit  costs. 

SPA  WAR  is  currently  developing  and  testing  improvements  to  system 
performance. 

The  Command  and  Control  Processor  acquires  information  from  other 
communications  systems,  stores  the  information,  and  reformats  it  for  use 
by  the  Aegis  combat  system  on  aircraft  carriers  and  naval  combat  surface 
ships.  Although  SPA  WAR  originally  approved  a  small  purchase  of  the 
processor  during  the  low-rate  initial  production,  three  subsequent  low-rate 
initial  production  increases  bought  the  total  to  41  percent  of  the  inventory 
objective.  From  1995  to  2000,  there  were  263  problems  noted  with  the 
system,  mostly  involving  software,  during  battle  group  system  integration 
tests.  The  processor  has  severe  suitability  problems  because  it  breaks 
down  impredictably  up  to  12  hours  at  a  time  (due  to  software  problems) 
and  fi-eezes  up,  which  eliminates  the  system’s  capability  to  provide  current 
situational  awareness.  Also,  on  an  aircraft  carrier  that  we  visited, 
operators  said  that  the  processor  would  not  integrate  with  other  systems, 
even  though  it  is  designed  to  do  so.  To  prevent  the  breakdowns,  SPAWAE 
developed  a  workaround  procedure,  which  involves  resetting  the  system 
every  2  hours  instead  of  every  24  hours.  In  addition,  the  breakdowns  and 
workaroimds  put  more  pressure  on  operators  and  maintainers  during 
combat  or  hostile  situations. 
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Some  of  SPAWAR’s  13  other  systems  with  a  high  percentage  of  low-rate 
initial  production  buys  also  had  operational  effectiveness  and  suitability 
problems.  The  Pacific  Fleet  experienced  46  problems  with  the  Navy 
Extremely  High  Frequency  Satellite  Communications  System  from  1995  to 
2000.  The  problems  involved  hardware  and  software,  interoperability,  and 
training.  For  example,  in  February  2000,  system  performance  was 
degraded  due  to  a  part  failure.  In  April  2000,  SPAWAR  reported  that  the 
problem  had  been  solved.  However,  about  a  month  later,  the  system  had 
problems  again,  resulting  in  no  response  from  the  satellite  and  time¬ 
tracking  errors  being  returned.  The  system  was  again  placed  in  a  degraded 
status  and,  as  of  August  2000,  the  problem  was  still  unresolved.  The 
Pacific  Fleet  also  experienced  problems  with  the  High  Frequency  Radio 
Group,  mainly  due  to  system  performance  problems  and  training 
shortfalls.  For  example,  on  a  ship  visit  in  October  2000,  ship 
communications  personnel  said  that  the  system  had  broken  down  several 
times  for  a  duration  of  1  week  to  1  month  at  a  time.  They  said  that,  when  it 
breaks  down,  the  operators  must  tune  in  the  radio  frequency  manually,  but 
ship  operators  have  not  been  trained  to  do  this  because  they  were  used  to 
relying  on  the  system  to  tune  into  a  particular  frequency  automatically. 

The  Pacific  Fleet  also  identified  several  other  problems  with  the  radio 
group,  including  a  ship  that  experienced  15  failures  of  a  system  switch 
within  11  months. 

The  SPAWAR  commander  said  that  the  Command  did  not  have  adequate 
controls  and  oversight,  at  the  time  of  most  of  these  low-rate  initial 
production  decisions,  to  either  mitigate  or  manage  risks  associated  with 
procuring  and  fielding  large  percentages  of  systems  during  low-rate  initial 
production.  He  said  the  need  for  more  discipline  in  the  acquisition  process 
contributed  to  the  interoperability,  performance,  and  suitability 
deficiencies  we  identified.  He  further  noted,  however,  that  some  problems 
are  part  of  the  cost  of  doing  business  with  new  systems  and  are  worth  the 
risk  to  provide  systems  to  the  fleet  quickly.  According  to  the  SPAWAR 
commander,  the  most  meaningful  measure  of  success  is  whether  the 
systems  are  meeting  their  operational  requirements,  and  he  said  that 
SPAWAR’s  systems  are  meeting  theirs  based  on  a  performance  parameter 
called  operational  availability.®  However,  according  to  the  DOD  Director  of 


®  Operational  availability  is  calculated  in  two  ways.  For  continuously  operating  systems,  it 
is  calculated  by  dividing  the  amount  of  time  that  the  system  is  up  and  running  by  the 
cumulative  amount  of  time  that  the  system  is  up  and  running  plus  the  down  time.  For  on- 
demand  systems,  operational  availability  is  calculated  by  dividing  the  niunber  of  times  the 
system  was  available  by  the  number  of  times  the  system  was  required  to  be  available. 
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Navy  and  SPAWAR 
Take  Steps  to  Reduce 
Low-Rate  Initial 
Production  Risks 


Operational  Test  and  Evaluation  and  the  Chief  of  Naval  Operations’  Office 
of  Test,  Evaluation,  and  Technology  Requirements,  operational  availability 
is  only  one  of  a  number  of  key  performance  measures,  and  an  overall 
assessment  of  system  performance  should  not  be  based  solely  on  that 
parameter. 


The  Navy  and  SPAWAR  have  taken  or  plan  to  take  a  number  steps  to 
mitigate  the  risks  of  large  low-rate  initial  production  procurements.  To  add 
more  discipline  and  rigor  to  the  low-rate  initial  production  decision 
process,  the  Command  now  requires  program  managers  to  use  a 
standardized  checklist  and  report  template  as  part  of  reviewing  and 
approving  low-rate  initial  production  purchase  requests.  SPAWAR  has  also 
established  an  Acquisition  Reform  Office  to  serve  as  a  focal  point  and 
command-wide  disseminator  of  lessons  learned  and  process 
improvements.  The  reform  office  is  currently  developing  a  “Rules  of  the 
Road”  Acquisition  Guidebook  for  SPAWAR  program  managers.  Further,  in 
discussions  of  our  findings  and  observations  during  this  review,  the 
commander  called  for  the  development  of  risk  management  guidance  for 
information  systems  and  agreed  to  suggested  improvements  in 
documenting  and  justifying  low-rate  initial  production  decisions. 

The  SPAWAR  commander  said  better  risk  management  guidance  would 
improve  low-rate  initial  production  decisions  on  information  systems, 
especially  when  milestone  decision  authorities  and  program  managers 
rotate  in  and  out  over  time.  The  commander  stated  that  he  and  the 
program  managers  primarily  use  their  acquisition  knowledge,  wisdom,  and 
experience  when  making  risk  management  decisions. 

In  discussions  with  Navy  and  SPAWAR  officials,  we  noted  that  the 
Acquisition  Decision  Memorandums,®  used  to  document  and  support 
milestone  decisions,  did  not  always  include  the  low-rate  initial  production 
quantify  being  approved,  the  cumulative  number  of  low-rate  initial 
production  items  that  had  been  approved,  or  the  cumulative  low-rate 
initial  production  percentage.  We  also  noted  that,  at  SPAWAR,  the 
justification  for  approving  low-rate  initial  production  purchases  was  not 
always  documented.  The  Navy  and  SPAWAR  officials  agreed  to  document 


®  The  Acquisition  Decision  Memorandum  is  a  memorandum  signed  by  the  milestone 
decision  authority  that  documents  decisions  made  as  a  result  of  a  milestone  decision 
review  or  an  in-process  review.  DOD  Instruction  5000.2  and  Navy  Instruction  5000.2B 
require  a  decision  memorandum  for  all  acquisition  milestones  for  major  and  non-major 
programs  but  do  not  specify  format  or  content. 
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in  the  Acquisition  Decision  Memorandum  the  justification  for  all  low-rate 
initial  production  approvals,  as  well  as  the  current  inventory  objective  and 
the  cumulative  number  of  units  bought  imder  low-rate  initial  production. 
SPAWAR  also  agreed  to  include  in  its  quarterly  program  status  report 
when  cumulative  low-rate  initial  production  approvals  reach  50  percent  or 
more  of  the  current  inventory  objective.  Recognizing  that  other  Navy 
activities  can  benefit  from  the  low-rate  initial  production  decision 
checklist  and  report  template,  we  recommended  that  this  guidance  be 
distributed  throughout  the  Navy.  The  Navy  subsequently  distributed  the 
guidance  DOD-wide.  Finally,  the  Navy  and  SPAWAR  agreed  to  supplement 
acquisition  training  for  program  managers  and  staff  by  incorporating  risk 
management  tools  into  existing  combes. 

Conclusions 

In  seeking  to  provide  new  information  systems  to  the  fleet  as  quickly  as 
possible,  SPAWAR  officials  procured  and  fielded  relatively  large  quantities 
of  systems  during  low-rate  initial  production  and  before  completing 
operational  testing.  Our  subsequent  review  of  seven  of  these  systems 
found  that  six  had  experienced  operational  problems  that  negatively 
impacted  the  fleet.  The  SPAWAR  commander  noted  that  controls  and 
oversight,  at  the  time  of  most  of  these  decisions,  were  not  adequate  to 
either  mitigate  or  manage  risks  associated  with  procuring  and  fielding 
large  percentages  of  systems  during  low-rate  initial  production.  He  said 
the  need  for  more  discipline  in  the  acquisition  process  contributed  to 
deficiencies  we  identified.  Since  that  time.  Navy  and  SPAWAR  officials 
have  taken  or  have  plans  to  take  a  number  steps  to  mitigate  the  risks  of 
large  low-rate  initial  production  procurements.  In  addition,  they  have 
agreed  to  implement,  and  in  one  case  have  already  implemented,  process 
improvements  we  suggested  during  the  course  of  this  review.  Given  these 
actions,  we  are  not  making  any  recommendations  in  this  report. 

Agency  Comments 

In  commenting  on  our  draft  report,  the  Navy  agreed  and  stated  that  the 
actions  taken  and  planned  by  it  and  the  SPAWAR  Command  are  expected 
to  improve  the  Navy’s  low-rate  initial  production  decision  process.  The 
Navy’s  comments  appear  in  appendix  I. 

Scope  and 
Methodology 

To  acquire  information  about  the  number  and  status  of  SPAWAR  low-rate 
initial  production  programs,  we  interviewed  officials  and  obtained 
documentation  from  the  SPAWAR  Acquisition  Reform  Office;  selected 
SPAWAR  program  offices;  the  Office  of  the  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition);  the  Office  of  the  Deputy 
Assistant  Secretary  of  the  Navy  (Command,  Control,  Communications, 
Computers,  and  Information/Electronic  Warfare/Space);  the  Office  of  the 
Deputy  Director  Defense  Procurement  Strategies;  the  Office  of  the  Under 
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Secretary  of  Defense  (Acquisition  and  Technology);  and  the  Office  of  the 
Chief  of  Naval  Operations. 

To  obtain  detailed  information  about  the  impact  of  the  high  percentage  of 
SPAWAR  low-rate  initial  production  procurements,  we  selected  and 
reviewed  14  programs,  representing  70  percent  of  all  SPAWAR  low-rate 
initial  production  programs.  Of  these  14  programs,  we  examined  8 
programs  in  detail  looking  at  operational,  logistic,  interoperability,  and 
training  issues  to  determine  how  well  these  programs  were  performing. 

To  obtain  information  about  the  operational  testing,  evaluation, 
interoperability,  and  fielding  of  low-rate  initial  production  systems,  we 
interviewed  officials  and  obtained  documentation  firom  the  Office  of  the 
Director  of  Navy  Test,  Evaluation,  and  Technology  Requirements;  the 
Office  of  the  Navy  Commander  Operational  Testing  and  Evaluation  Force; 
the  Office  of  the  Program  Manager  for  Battle  Group  Systems  Integration 
Testing;  and  the  Office  of  the  Director  of  Operational  Test  and  Evaluation, 
Office  of  the  Assistant  Secretary  of  Defense. 

To  obtain  information  about  the  operation,  performance,  interoperability, 
maintenance,  repair,  retrofit,  suitability,  and  training  regarding  low-rate 
initial  production  systems  in  the  fleet,  we  visited  the  Naval  Surface  Force 
Command,  Pacific;  the  Naval  Air  Command,  Pacific;  and  the  Naval 
Submarine  Command,  Pacific,  Squadron  Eleven.  We  also  visited  specific 
ships  in  each  command,  including  the  U.S.S.  Benfold (DDG-65),  the  U.S.S. 
Pear/ (LSD-52),  the  U.S.S.  John  C.  5ifenn/s  (CVN-74),  and  the  U.S.S. 
Salt  Lake  C/i^(SSN-716).  In  addition,  we  held  discussions  with  SPAWAR 
program  officials  and  officials  from  SPAWAR’s  In-Service  Engineering 
Activity,  Fleet  Support  Engineering  Team,  and  Installations  and  Logistics 
Directorate. 

To  obtain  information  about  the  laws,  regulations,  procedures,  and 
guidance  governing  the  procurement  of  information  technology  systems  in 
low-rate  initial  production,  we  interviewed  officials  and  obtained 
documentation  from  the  Office  of  the  Commander,  Space  and  Naval 
Warfare  Systems  Command;  the  Office  of  the  Chairman  Deskbook 
Working  Group  (DOD  5000  Rewrite);  the  Assistant  Secretary  of  the  Navy 
(Research,  Development  and  Acquisition);  and  the  Office  of  the  Assistant 
Deputy  Under  Secretary  of  Defense  (Systems  Acquisition).  We  also 
reviewed  selected  laws  and  regulations  governing  low-rate  initial 
production,  including  title  10  of  the  U.S.  Code,  the  DOD  5000  series 
acquisition  regulations  (thel996  and  revised  2000  version),  and  Secretary 
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of  the  Navy  Instruction  5000-2B  governing  acquisition  and  procurement  of 
low-rate  initial  production  and  commercial-off-the-shelf  technology. 

We  conducted  our  review  from  Jxme  2000  through  May  2001  in  accordance 
with  generally  accepted  government  auditing  standards. 

We  are  sending  copies  of  this  report  to  interested  congressional 
committees;  the  Honorable  Donald  H.  Rumsfeld,  Secretary  of  Defense;  and 
the  Honorable  Mitchell  E.  Daniels  Jr.,  Director  of  the  Office  of 
Management  and  Budget.  Copies  will  be  made  available  to  others  upon 
request. 

Please  contact  me  at  (202)  512-4821  if  you  have  any  questions  regarding 
this  report.  Key  contributors  to  this  report  were  Cristina  Chaplain,  Joe 
Dewechter,  Dorian  Dimbar,  Stephanie  May,  Gary  Middleton,  Sarah 
Prehoda,  Richard  Price,  and  William  Woods. 


Sincerely  yours. 


James  F.  Wiggins 
Director 

Acquisition  and  Sourcing  Management 
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Appendix  I:  Comments  From  the  Department 
of  the  Navy 


DEPARTMENT  OF  THE  NAVY 

OFFICE  OF  THE  ASSISTANT  SECRETARY 
RESEARCH,  DEVELOPMENT  AND  ACQUISITION 
1000  NAVY  PENTAGON 
WASHINGTON  DC  20350-1000 

JUl  1 3  2001 

Mr.  James  F.  Wiggins 
Director 

Acquisition  and  Sourcing  Management 
U.S.  General  Accounting  Office 
Washington,  D.C.  20543 

Dear  Mr.  Wiggins: 

This  is  the  Department  of  Navy  response  to  the  General 
Accounting  Office  {GACi  draft  report,  "DEFENSE  ACQUISITIONS: 
Actions  to  Improve  Navy  SPAWAR  Low-Rate  Initial  Production 
Decisions"  dated  June  14,  2001  (GAO  Code  707533/OSD  Case  4019). 

The  Navy  concurs  with  the  GAO  report.  Changes  to 
procedures  agreed  to  by  the  GAO  and  the  Navy  during  the  course 
of  the  audit  are  expected  to  improve  the  Navy's  Lov-?  Rate  Initial. 
Production  decision  process. 

A  summary  of  the  agreed  upon  changes  to  procedures  and  an 
emphasis  on  training  are  enclosed. 


Sincerely, 


Paul  A.  Schneider 
Principal  Deputy 


Enclosure 
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Appendix  I:  Comments  From  the  Department 
of  the  Navy 


Enclosure 

Department  of  Defense  Response  to  General  Accounting  Office 
Draft  Report: 

DEFENSE  ACQUISITIONS:  Actions  to  Improve  Navy  SPAWAR  Low-Rate 
Initial  Production  Decisions _ 


This  is  a  summary  of  the  agreed-upon  changes  that  will  improve 
the  LRIP  performance  of  fielded  systems  at  less  cost  or 
disruption  to  fleet  operations: 

1)  Acquisition  Decision  Memoranda^ 

The  Navy  agreed  to  require  that  all  program  Acquisition  Decision 
Memoranda  -  major  and  non-major  -  include  the  justification  for 
the  LRIP  decision,  together  with  cumulative  units  approved  and 
the  percent  LRIP  represents  to  the  total  inventory  objective. 

2)  Operations  and  Maintenance  Training 

The  Navy  agreed  to  better  define  O&M  training  responsibilities 
between  the  program  manager  and  the  fleet  for  systems  fielded 
during  LRIP  when  they  will  exceed  ten  percent  of  the  expected 
production  quantity. 

3)  Risk  Management  (and  Earned  Value)  Guidance 

The  Navy  recogniaes  the  need  to  reinforce  risk  management 
training  that  our  program  managers  and  program  management  staff 
receive  as  part  of  their  program  management  certification 
training  through  the  Defense  Acquisition  University's  Defense 
Systems  Management  College.  ASN(RDA)  will  remind  senior 
acquisition  officials  of  the  tools  available  to  help  identify 
risks  and  to  develop  mitigation  measures  at  the  Navy  Acquisition 
Reform  Senior  Oversight  Council  (NARSOC)  meeting  scheduled  for 
the  second  quarter  of  fiscal  year  2002.  In  the  meantime,  risk 
management  issues  will  be  incorporated  into  Information  Alerts, 
which  are  periodic  bulletins  from  the  Department  of  the  Navy 
Acquisition  Reform  Executive  and  are  provided  directly  to 
acquisition  workforce  members.  These  Info  Alerts  will  address 
the  use  of  independent  technology  readiness  assessments, 
independent  expert  reviews  of  software  intensive  programs,  and 
earned  value  management,  as  well  as  risk  management  measures. 
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Appendix  I:  Comments  From  the  Department 
of  the  Navy 


4)  Independent  Technology  Readiness  Assessment 

Regarding  independent  technology  readiness  assessments  and 
Independent  expert  review  of  ACAT  programs,  the  I'lavy  will 
continue  to  meet  the  requirements  of  DODI  5000.2,  paragraph 
4. 7. 3. 2. 3. 1.2,  and  DOD  Regulation  5000~2R,  paragraphs  C2.6.8  and 
C7,5.3,  to  employ  independent  reviews.  The  Department  of  the 
Navy  Science  and  Technology  Executive  (Department  of  the  Navy 
Chief  Technology  Officer)  will  oversee  these  assessments  and 
review’s . 


(707633) 
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The  first  copy  of  each  GAO  report  is  free.  Additional  copies  of  reports  are 
$2  each.  A  check  or  money  order  should  be  made  out  to  the 
Superintendent  of  Documents.  VISA  and  MasterCard  credit  cards  are  also 
accepted. 

Orders  for  100  or  more  copies  to  be  mailed  to  a  single  address  are 
discounted  25  percent. 

Orders  by  mail: 

U.S.  General  Accounting  Office 
P.O.  Box  37050 
Washington,  DC  20013 

Orders  by  visiting: 

Room  1100 

700  4*  St,  NW  (comer  of  4*"  and  G  Sts.  NW) 

Washington,  DC  20013 

Orders  by  phone: 

(202)  512-6000 
fax:  (202)  512-6061 
TDD  (202)  512-2537 

Each  day,  GAO  issues  a  list  of  newly  available  reports  and  testimony.  To 
receive  facsimile  copies  of  the  daily  list  or  any  list  from  the  past  30  days, 
please  call  (202)  512-6000  using  a  touchtone  phone.  A  recorded  menu  will 
provide  information  on  how  to  obtain  these  lists. 

Orders  by  Internet 

For  information  on  how  to  access  GAO  reports  on  the  Internet,  send  an  e- 
mail  message  with  “info”  in  the  body  to: 

Info@www.gao.gov 

or  visit  GAO’s  World  Wide  Web  home  page  at 
http://www.gao.gov 


Ordering  Information 


To  Report  Fraud, 
Waste,  and  Abuse  in  • 
Federal  Programs  ! 


Contact  one: 

Web  site:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

1-800-424-5454  (automated  answering  system) 
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